Confidence-rated Regression by Localized Median Boosting

In this paper we describe and analyze LOCMEDBOOST, an algorithm that boosts regressors with input dependent weights. The algorithm is a synthesis of median boosting [1] and localized boosting [2, 3, 4], and unifies the advantages of the two approaches. We prove boostingtype convergence of the algorithm and give clear conditions for the convergence of the robust training error, where robustness is measured in terms of the expert population and with respect to the underlying confidence estimate. We extend Rätsch and Warmuth’s results [5] on efficient margin maximization to show that the algorithm can converge to maximum achievable margin in a finite number of steps. We also extend probabilistic bounds on the generalization error derived for ADABOOST. The results provide bounds on the confidence-interval-type error and qualitatively justify the algorithmic objective of the minimization of the robust error. Finally, we present some promising experimental results on synthetic and benchmark data sets.